Evaluation of five commercial assays for detecting HIV 1 & 2 antibodies, Addis Ababa

The major operational characteristics of five commercially available assays for the detection of antibodies to Human Immunodeficiency Virus (HIV1 & 2) were evaluated. Four Enzyme Linked Immuno-sorbent assays (ELISAs) and one simple immuno-dot assay with visual reading, were assessed using a panel of 265 sera (18.8% hospital suspected patients, 18.8% commercial sex-workers (CSW), 31.5% blood donor sample (BDS), and 30.9% of them were scholarship winners (SSW)). Sensitivity, specificity, positive predictive value, test efficiency, delta (δ) values (for the four ELISAs) were determined. All the assays had higher sensitivities (98.7-100%), specificities (97.2-99.1%), and test efficiencies (98.1-99.6%). Higher positive and negative delta (δ+,δ -) values, +1.17 and –0.99, were observed for ICE*HIV 1-0-2 and Vironostika Uniform II PLUS O, respectively. HIV-SPOT HIV 1 & 2 showed highest value of ease of performance and suitability for small blood bank collection centers. Results of this study showed that the test efficiency, sensitivity, and specificity of the test kits were excellent as compared to the reference test. Further studies on cost-effectiveness and evaluation of newly arrived test kits before use at different levels are recommended. [Ethiop. J. Health Dev. 1999;13(2):175-180] Introduction The first Enzyme Linked Immuno Sorbent Assay (ELISA) for antibodies to Human Immunodeficiency Virus (HIV) were manufactured by coating purified HIV lysate on to the surface of micro-titration plates or beads (First Generation Assays). Later on, ELISAs have been developed which use antigens of either HIV recombinant polypeptide or synthetic peptide (Second or Third generation Immuno assays) (1-3). The use of the third generation immunoassay for the detection of HIV has reduced the interval between infection and antibody detection. These assays detect antibody to HIV earlier than the first and second generation assays including Western Blot (WB) from serum and urine (4-6). Incomplete cross-reactivity between HIV-1 and HIV-2 needs to have a combination of assays with acceptable sensitivity and specificity for both viruses. Even with HIV antibody screening, assays that have excellent sensitivity and specificity, false-positive results can not be ruled out, especially when used in a population with low prevalence of HIV antibodies (1,7,8). However, ELISAs require 1.5 to 3.5 hours to perform and need sophisticated and expensive equipment. This makes the assays technologically inappropriate for use in small laboratories in developing countries. On the other hand, simple immuno dot assays for HIV have been developed that do not require much equipment and that yield results after a few minutes (1,2,3,9). In countries like ours, where resource is scarce, the need for less expensive, more reliable and simpler assays for the detection of HIV antibody is very important. In the light of this, the present study attempts to investigate and evaluate an assay which gives the most reliable result with a good test performance, relatively lower price, and less complexity. Accordingly, five commercial HIVantibody assays were evaluated at the Ethiopian Health and Nutrition research Institute (EHNRI), National Referral Laboratory for AIDS (NRLA). _______________________________________ Ethiopian Health and Nutrition Research Institute, P.O. Box 1242, Addis Ababa, Ethiopia. Methods In the evaluation of the assays for the detection of antibodies for HIV, a panel of 265 human sera was tested, 18.8% were hospital suspected patients, 18.8% commercial sexworkers (CSW), 31.5% blood donor samples (BDS), and 30.9% were Scholarship winners (SSW). A series type of test was conducted and the results obtained using a combination of assays were compared with those obtained using HIV-1 Western Blot (Genelabs Diagnostics), which is used as a reference (gold standard) test (1,3,10-12). Of these samples 57.7% were Western Blot (WB) (HIVBLOT 2.2, Genelabs, Diagnostics, Singapore) reactives, 1.5% were WB indeterminate, and 40.8% were nonreactive samples. The interpretation of the result for WB was according to the criteria given by the American Red Cross Society (3). The prevalence of HIV among SSW, BDS,CSW was reported as 5.3% (13), 6.6% (14), and 47-59% (15), respectively. Originally, these samples were sent to NRLA for confirmation and diagnosis of HIV infection. Five test kits, viz., HIV-SPOT HIV 1 & 2, Vironostika Uniform II PLUS 0 HIV 1 & 2, ICE* HIV 1-0-2, Innotest HIV 1 & 2 and RecombigenR EIA HIV 1 & 2 combination assays, which were available in NRLA were evaluated. Three of the ELISAs (Vironostika Uniform II PLUS 0, Innotest HIV 1 & 2 and ICE* HIV 1-0-2,) were third generation, while RecombigenR HIV 1 & 2 EIA was a second generation test kit (2). All the test kits were combination assay types for the detection of HIV 1 & 2. Their characteristics are summarized in Table1. The tests were performed according to the manufacturers’ instructions. In short, weak reactivity for HIV SPOT HIV 1 & 2 was evaluated visually according to the criteria given in the manufacturer’s manual, while for the other test kits, optical density (OD) reading just above the cut-off value was used. Except for HIV-SPOT HIV 1 & 2 test kit, all others require the following equipment and reagents in the laboratory: automatic washer, spectrophotometer reader, water bath, refrigerators for storage of test kits, incubation boxes, stop solutions (sulfuric acid), pipettes (multi channel and single channel), micro titration plates, dilution tubes, racks, distilled water, agitator, aspiration devices, incubators, troughs, graduated cylinders, pipette tips, distiller and thermal paper for spectro-photometer readers. Sensitivity (SEN), specificity (SPEC), positive predictive values (PPV), Test Efficiency (TE), Delta Values (δ+ and δ -, except for HIV SPOT HIV 1 & 2), false-positive ratio (FPR), the ease of performance and suitability for use in small blood bank collection centers for a particular HIV antibody assay were determined using the formula as described in Kerchoven, IV, et al. (1), Constantine, NT et al. (3), and Rose, NR, et al. (16). The statistical analyses were performed using the computer program STATA (Stata corporation, Texas, USA). Table 1: Characteristics of five commercial HIV antibody test kits evaluated in the study, Ethiopian Health and Nutrition Research Institute, National Referral Laboratory for AIDS, 1998. Paramete HIV-Spot HIV1 & 2 Vironostika Uniform II plus 0 Innotest HIV 1 & 2 ICE* HIV 1-0-2 RecombigenR EIA Manufacturer Genelabs Diagn Organon Teknika Innogenetics Murex Cambridge Test Type DOT IE IE IE IE Antigen Type SP/RP SP SP SP SP Coating on Membrane Flat wells Flat wells U-shaped wells Flat wells Cost USD/test** 1.50 0.50 0.50 0.50 0.50 Time for Test 10min 2 hours 2 hours 2 1⁄2 hours **Price is according to the World Health Organisation (WHO), 1998 SPSynthetic peptide, RP-Recombinat protein, IE-Indirect ELISA Table 2: Sensitivity (SEN%), specificity (SPEC %), positive predictive value (PPV %), test efficiencty (TE %), and Delta values (δ+δ) of the test kits, Ethiopian Health and Nutrition Research Institute, National Referral Laboratory for AIDS, 1998. Parameter HIV-SPOT HIV1 & 2 VIRONOSTIKA UNIFORM II PLUS 0 I NOTEST HIV 1 & 2 ICE* HIV 1-0-2 RECOMBIGENR EIA SEN 98.69(95.36-99.84) 100(97.62-100) 98.69(95.36-99.84) 100(97.62-100) 98.69(95.36-9.84) SPEC 8.15(93.47-99.77) 99.07(94.95-99.97) 97.22(92.09-99.42) 97.22(92.09-99.42) 8.15(93.47-99.77) PPV 98.69(95.36-99.84) 99.35(96.43-99-98) 98.05(94.41-99.59) 98.07(94.48-99.68) 8.69(95.36-99.84) TE 98.47(96.12-99.58) 99.62(97.88-99.99) 98.08(95.58-99.37) 98.85(96.68-99.76) 8.47(96.12-99.58) δ+ NC 0.438 0.538 1.176 0.997 δ NC -0.990 -0.190 -0.080 -0.620 **-95% confidence interval NC-Not calculated Results All the assays evaluated in the study had good sensitivities, specificities, positive predictive values and test efficiencies with ranges of 98.7-100%, 97.2-99.1%, 98.1-99.4% and 98.1-99.6%, respectively. The positive and negative delta values (δ+ and δ -) ranged from 0.438 to 1.176 and –0.99 to –0.08, respectively (Table 2). The rate of false-positivity (FPR) were found to be 0.93%, 1.85% and 1.85%, 2.78% and 2.78% for Vironostika Uniform II PLUS 0, RecombigenR HIV 1 & 2, HIV SPOT HIV 1 & 2, Innotest HIV 1 & 2 and ICE* HIV 1-0-2 test kit, respectively, while the false negativity rates were 0 and 1.3%. Unlike the other four ELISAs, HIV-SPOT (rapid test) was found to be very easy and very suitable in its ease of prformance and suitability for use in small blood bank collection centers (Table 3). Four indeterminate samples (1.5% out of the total), which were determined by Western Blot, were included in the evaluation of these test kits. At least two of the test kits were weakly reactive towards one of the indeterminate samples. However, none of these samples was reactive towards Vironostika Uniform II PLUS 0 HIV1 & 2 test kit (Table 4). Table 3 : Total sera tested (Positives and Negatives) and calculated true positives, true negatives, ease of performance and suitability for use in small blood bank collection centers (SSBBCC) of the evaluated test kits, Ethiopian Health and Nutrition Research Institute, National Referral Laboratory for AIDS, 1998. HIV-SPOT HIV1 & 2 Vironostika Uniform II Innotest Plus 0 HIV1 & 2 ICE* HIV 1-0-2 Rerecombigen R EIA Total Negatives Samples 108 108 108 108 108 Total Positives Samples 153 153 153 153 153 True Negatives 106 107 105 105 106 True Positives 151 153 151 153 151 Ease of Performance VE LE LE LE LE SSBBCC VS LS LS LS LS VE-Very easy, LE-Less easy, VS-Very suitable, LS-Less suitable Discussion As already showen by different researchers (1,3,9) and from our laboratory experience, the conventional method of ELISA exhibited a number of shortcomings. The assay requires instrumentation and preparation of reagents; it is not rapid and, as a result, is neither easy to perform nor suitable (ease of performance and suitability test) in places where the time gap between blood donation and transfusion is very short. Moreover, in resource-poor settings Table 4: Test results of the indeterminate samples by the evaluated test kits, Ethiopian Health and Nutrition Research Institute, National Referral laboratory for AIDS, 1998. Sample Code Weakly Reactive by Result of WB Bands observed L-161 Recombigen HIV 1⁄2 EIA Indeterminate P24(weak) L-187 HIV-SPOT, Innotest, ICE*HIV 1-0-2 Indeterminate P24,P66(both weak) L-261 Innotest, ICE* HIV 1-0-2 Indeterminate gp41(weak) L-263 HIV-SPOT, Recombigen HIV Indeterminate gp41(weak) Positive control Reactive All bands observed Negative control Non-reactive No bands observed where frequent electric power interruptions and shortage of distilled water exist, there will be a negative effect on the performance of the test. However, ELISAs are preferable to screen large number of samples as compared to rapid and confirmatory assays (3,17). In contrast, though immuno-dot blot assays are more expensive than ELISAs, they are recommended in rural areas (in field), blood banks, and emergency rooms where there are shortages of water supply and electricity, as these tests are easier to perform, yield result after few minutes and easy to interpret in such conditions (10,12,18-22). The sensitivities, specificities, and positive predictive values of the test kits were high, ranges of 98.7-100%, 97.2-99.1%, and 98.1-99.4%, respectively. The highest positive delta value was observed for ICE* HIV 1-0-2 test kit, while the highest negative delta value was observed for Vironostika Uniform II PLUS 0 HIV 1 & 2 test kit. The test performance of these kits (test efficiency), which is dependent on both specificity and sensitivity, was higher for all the kits and was found to be highly satisfactory, being in the range of 98.1-99.6%. The delta (δ) values provide statistical estimates of the test sensitivity and specificity and permit differentiation between ELISAs of similar sensitivity and specificity and help to see the comparison of the efficacy of ELISAs to separate the negative and positive antibody serum populations from the cut-off value. It also reflects the ability of an ELISA test to produce consistently high sample/ cut-off ratios; sample optical density (OD) ratio lie far above or below the cut-off OD for HIV-antibody positive and negative sera, respectively. The higher the positive and negative delta values, the greater the probability that the test will correctly identify antibodypositive and antibody-negative sera (1,2). The higherst δ+ value observed for ICE * HIV 1-0-2 (δ+ =+1.77) shows that this test kit has the ability to characterize the positive samples that lie far above the cut-off value. This is further demonstrated by the fact that the kit has the highest false-positivity ratio. On the other hand, the higherst negative delta value was obtained for Vironostika Uniform II PLUS 0 test kit (δ-=-0.99), which showed the assay has a greater margin for variation in test results without the occurrence of more false positive results (FPR = 0.93%), and this increases the confidence in the specificity estimates. The sensitivity of Vironostika Uniform II PLUS 0 and ICE* HIV 1-0-2 was high (100% in each case) compared to the others. This concurred with the absence of false-negative results in both cases which make these types of ELISAs feasible for screening of large samples as elaborated by different researchers (2,6,17). Eventhough all tested kits had high sensitivities and specificities, Vironostika Uniform II PLUS 0 characterized negative samples in a better way than the other test kits and had a clear margin for separation of the sample OD ratio that lies below the cut-off value. Eventhough the observed test efficiencies were high (98.1-99.6%), Virononstika Uniform II PLUS 0 showed a high test performance (TE, 99.6%) as compared to the other kits which showed better sensitivity and specificity. In general, all the test kits evaluated in the study had good sensitivity, specificity, positive predictive value and test efficiency as compared to the reference test, Western Blot. The ease of performance and suitability for use in small blood bank collection centers was higher for HIVSPOT (rapid test) which indicates that this test kit is very simple to use in resource limited areas and in emergency conditions as also recommended by Constantine, NT,1993 and Myrmel, H, 1990. In addition, the study will give comparative data to enable the users to arrive at a decision of their own, depending on their needs and conditions, to choose the appropriate sensitive and specific test kits for screening and combinations of ELISAs for confirmation of HIV-infection. Recommendations Since the development, introduction, and use of the newly arriving HIV antibody detection assays is a dynamic process, the evaluation of these assays before use at different levels and condition is essential. The sensitivity, specificity, predictive values, falsepositive ratio, ease of performance, suitability for use in small blood bank collection centers, test efficiency and delta values need to be evaluated using different combinations of assays and ELISA systems to confirm antibodies to HIV infetion. Further study on the cost-effectiveness of these and related HIV antibody detection assays is recommended in order to adopt the most cost-effective option. Acknowledgements We greatly acknowledge the Department of Epidemiology and AIDS, AIDS/STD Team, MOH, and the Ethio-Netherlands AIDS Research Project (ENARP) in EHNRI for providing us with the ELISA and Western Blot test kits, respectively. The valuable comments and suggestions on thearticle by Dr. Arnaud Fontanet and Ato Tesfaye Mengesha are highly appreciated. Finally, EHNRIis aknowledged for providing us with all the required laboratory facilities. References1. Kerchoven IV, Vercauteren F, Piot P, and Groen,GVD. Comparative evaluation of 36 commercial assays for detecting antibodies to HIV. Bull WHO 1991;69(6):753-60.2. Constantine NT, Collahan JD, Watts DM. Retroviral Testing, Essentials for Quality Controland Laboratory Diagnosis.2nd edn. Boca Raton: CRC Press Inc, 1992:35.3. Constantine NT. Serologic tests for the retroviruses:approaching a decade of evolution. AIDS.1993;7:1-13. 4. Zaaijer HL, Oehlers PVE, Kraaijeveld T, Altena E, Lelie PN. Early detetion of antibodies toHIV-1 by third generation assays. Lancet 1992; 340:770-2.5. Stetler HC, Granade TC, Nunez CA, Meza R, Terrell S, Amador L, George JR. Fieldevaluation of rapid HIV serologic tests for screening and confirming HIV-1 infection in Honduras.AIDS 1997; 11:369-75.6. Desai S, Bates H, Michalski FJ. Detection of antibody to HIV-1 in urine. Lancet1991;337:183-4.7. Messele T, Zewdie D. Comparison of two generations of Wellcozyme kits. Ethiop J Health Dev1990;4(2):207. 8. Constantine NT, Fox E, Abatte EA, Woody JN. Diagnostic usefulness of five screening assaysfor HIV in an East African city where the prevalence of infection is low AIDS 1989; 5:313-17. 9. World Health Oganization and United Nations Joint Program on HIV/AIDS, RevisedRecommendations for the selection and use of HIV antibody tests. WHO Wkly Epidemiol 1997;72:12, March 21, 81-8. 10. Groen GVD, Kerchoven IV, Vercauteren G, Piot P. Simplified and less expensive confirmatoryHIV testing Bull WHO 1991; 69(6):747-52. 11. Broder S, Merizan TC, Bolgnesi D. Text Book of AIDS Medicine. Williams & Wilkins,1994:149.12. Myrmel H, Holm-Hanson C, Haukenes G. Evaluation of two rapid tests for the detection of HIV-1 & HIV-2 antibodies. AIDS 1990;4(11): 1164-5.13. Sahlu T, Eshete H. The progression of HIV/AIDS in Ethiopia. Ethiop J Health Dev 1996;10:179-90.14. Fontanet AL, Messele T, Dejene A, Enquselassie F, Abebe A, Cutts FT, De Wolf F, Rinke DeWit TF, Sahlu T, Bindels P, Yeneneh H, Coutinho RA, Nokes DJ. Age-and sex-specific HIV-1 prevalence in the urban community setting of Addis Ababa, Ethiopia. AIDS 1998;12:315-22.15. Abebe A, Kuiken CL, Goudsmit J, Valk M, Messele T, Sahlu T, Yeneneh H, fontanet A, De Wolf F, Rinke De Wit TF. HIV Type 1 subtype C in Addis Ababa, Ethiopia. AIDS Res HumRetroviruses. 1997;13(12):1071-75.16. Rose NR, Friedman H, Fahey JL. Manual of Clinical Laboratory of Immunology, 3 rd edition, American Society for Microbiology, Washington DC, U.S.A. 1986:966.17. United Nations Joint Program on HIV/AIDS: Blood Safety and HIV, UNAIDS TechnicalUpdate (UNAIDS Best Practce Collection: Technical Update) Geneva; October 1997;3.18. Wilkson D, Wilkson N, Lombard C, Martin D, Smith A, Floyd K, Ballard R. Onsite HIVtesting in resource-poor settings: is one rapid test enough? AIDS 1997; 11:377-81.19. Parry JV, McAlpine L, Avillez MF. Sensitivity of six commercial enzyme immunoassay kitsthat detect both anti-HIV-1 and HIV-2. AIDS 1990; 4:355-60.20. Spielberg F, Kebeya CM, Ryder RW, Kifuani NK, Harris J, Bender TR, Heyward WL, QuinnTC. Field testing and comparative evaluation of rapid, visually read screening assays for antibodyto human immuno-deficiency virus. Lancet 1989;1:580-4.21. Kebeya CM, Bertozi S, Spielberg F, Ryder RW, Heyward WL, Quinn TC. Cost analysis ofdual rapid assay system for screening and confirming of HIV seropositivity. In: Abstracts from IV International conference on AIDS and associated cancers in Africa, Marseille, France, 18-20October, 1989. P.400, Abstract No.401.22. Constantine NT, Collahan JD, Watts DM. Time for HIV-1 & HIV-2 combinations tests? JVirol Methods 1989;26:219-22.